MAJORIZATION FOR CHANGES IN ANGLES BETWEEN 
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Abstract. Many inequality relations between real vector quantities can be succinctly expressed 
as "weak (sub)majorization" relations using the symbol -< m . We explain these ideas and apply them 
in several areas: angles between subspaces, Ritz values, and graph Laplacian spectra, which we show 
are all surprisingly related. 

Let 0(X,y) be the vector of principal angles in nondccreasing order between subspaces X and 
y of a finite dimensional space "H with a scalar product. We consider the change in principal angles 
between subspaces X and Z, where we let X be perturbed to give y. We measure the change using 
weak majorization. We prove that | cos 2 S(X, Z) — cos 2 &(y, Z)\^ w sin@{X , y), and give similar 
results for differences of cosines, i.e. | cos @(X, Z) — cos S(y, Z)\^ w sin @(X, y), and of sines, and of 
sines squared, assuming dimA* = dim^. 

We observe that cos 2 &(X, Z) can be interpreted as a vector of Ritz values, where the Rayleigh- 
Ritz method is applied to the orthogonal projector on Z using X as a trial subspace. Thus, our 
result for the squares of cosines can be viewed as a bound on the change in the Ritz values of an 
orthogonal projector. We then extend it to prove a general result for Ritz values for an arbitrary 
Hermitian operator A, not necessarily a projector: let A(PxA\x) be the vector of Ritz values in 
nonincreasing order for A on a trial subspace X, which is perturbed to give another trial subspace y, 
then |A (PxA\x) — A (-PyA|j;)| -< w (Amax — ^ m j n ) sin@(X, y), where the constant is the difference 
between the largest and the smallest eigenvalues of A. This establishes our conjecture that the root 
two factor in our earlier estimate may be eliminated. Our present proof is based on a classical but 
rarely used technique of extending a Hermitian operator in 7i to an orthogonal projector in the 
"double" space H 2 . 

An application of our Ritz values weak majorization result for Laplacian graph spectra compar- 
ison is suggested, based on the possibility to interpret eigenvalues of the edge Laplacian of a given 
graph as Ritz values of the edge Laplacian of the complete graph. We prove that J2 k \\\ -A 2 | < nl, 
where AjJ, and A 2 , are all ordered elements of the Laplacian spectra of two graphs with the same n 
vertices and with I equal to the number of differing edges. 
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graph vertex Laplacian, graph edge Laplacian. 
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1. Introduction. Many inequality relations between real vector quantities can 
be succinctly expressed as "weak (sub)majorization" relations using the symbol -< w 
that we now introduce. For a real vector x = [x\, • • • , x n ] let be the vector obtained 
by rearranging the entries of x in an algebraically non-increasing order. Vector y 
weakly majorizes vector x, i.e. x -< w y, if Ylt=i x \ — J2i=iUi: & = l,---,n. The 
importance of weak majorization can be seen from the classical statement that the 
following two conditions are equivalent: x^ w y and ^"=1 fifai) — S"=i 4>{l)i) f° r an 
nondccreasing convex functions cf>. Thus, a single weak majorization result implies a 
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great variety of inequalities. We explain these ideas and apply them in several areas: 
angles between subspaces, Ritz values, and graph Laplacian spectra, which we show 
are all surprisingly related. 

The concept of principal angles, also referred to as canonical angles, between 
subspaces is one of the classical mathematical ideas originated from Jordan with 
many applications. In functional analysis, the gap between subspaces, which is related 
to the sine of the largest principal angle, bounds the perturbation of a closed linear 
operator by measuring the change in its graph, while the smallest nontrivial princi- 
pal angle between two subspaces determines if the sum of the subspaces is closed. In 
numerical analysis, principal angles appear naturally to estimate how close an approx- 
imate eigenspace is to the true eigenspace. The chordal distance, the Frobenius norm 
of the sine of the principal angles, on the Grassmannian space of finite dimensional 
subspaces is used, e.g., for subspace packing with applications in control theory. In 
statistics, the cosines of principal angles are called canonical correlations and have 
applications in information retrieval and data visualization. 

Let Ti. be a real or complex n < oo dimensional vector space equipped with an 
inner product (x,y) and a vector norm ||a;|| = (x^x) 1 ^ 2 . The acute angle between two 
non-zero vectors x and y is defined as 

0(x,y)=arccosl%^ r G[O ) 7r/2]. 

IpIIIMI 

For three nonzero vectors x, y, z, we have bounds on the change in the angle 

\e(x,z)-e(y,z)\<6(x,y), (1.1) 

in the sine 

\sm(0(x, z)) - sm(0(y, z))\ < sm(0(x, y)), (1.2) 

in the cosine 

\cos{9(x,z))-cos(6(y,z))\ <sm(9(x,y)), (1.3) 

and a more subtle bound on the change in the sine or cosine squared 

|cos 2 (6»(a;,z)) -cos 2 (%,z))| = |sin 2 (6»(x, z)) - sin 2 (%, z))\ <sm{9{x,y)). (1.4) 

Let us note that we can project the space TL into the span{x,y,z} without changing 
the angles, i.e. the inequalities above present essentially the case of a 3D space. 

Inequality $F1$ is proved in Qiu et al. Q Th. 3.2, p. 514]. We note that Q 
follows from (|1.1|) . since the sine function is increasing and subadditive, see Qiu et al. 
p. 530]. 

It is instructive to provide a simple proof of the sine inequality H1.2[l using or- 
thogonal projectors. Let Px, Py, and Pz be, respectively, the orthogonal projectors 
onto the subspaces spanned by the vectors x, y, and z, and let || • |j also denote the in- 
duced operator norm. When we are dealing with ID subspaces, we have the following 
elementary formula sin(6*(a;, y)) = \\Px — Py\\ (indeed, P x — P y has rank at most two, 
so it has at most two non-zero singular values, but {P x — P y ) 2 x = (1 — \(x, y)\ 2 )x and 
(P x — P y ) 2 y = (1 — \(x, y)\ 2 )y for unit vectors x and y, so 1 — \(x, y)\ 2 = sin 2 (6'{a;, y}) 
is a double eigenvalue of (P x — P y ) 2 ). Then the sine l|1.2fl inequality is equivalent to 
the triangle inequality | \\P X - P z \\ - \\P y - P z \\ \ < \\P X - P y \\. 



MAJORIZATION: ANGLES, RITZ VALUES, GRAPH SPECTRA 



3 



In this paper, we replace ID subspaces spanned by the vectors x, y and z, with 
multi-dimensional subspaces X, y and Z, and we use the concept of principal angles 
between subspaces. Principal angles are very well studied in the literature, however, 
some important gaps still remain. Here, we are interested in generalizing inequalities 
H1.2[l - i|1.4|) above to multi-dimensional subspaces to include all principal angles, using 
weak majorization. 

Let us denote by Q(X,y) the vector of principal angles in nondecreasing order 
between subspaces X and y. Let dimA = dim} 7 , and let another subspace Z be 
given. We prove that | cos 2 0(A, Z) — cos 2 Q(y, Z)\~< w sin ©(A, y), and give similar 
results for differences of cosines, i.e. | cosO(A, Z) — cosQ{y, Z)\-i w sin 6 (A", y), and 
of sines, and of sines squared. This is the first main result of the present paper, see 
Sectional The proof of weak majorization for sines is a direct generalization of the ID 
proof above. Our proofs of weak majorization for cosines and sines or cosines squared 
do not have such simple ID analogs. 

Pioneering results using angles between subspaces in the framework of unitarily 
invariant norms and symmetric gauge functions, equivalent to majorization, appear 
in Davis and Kahan [2|, which introduces many of the tools that we use here. The 
main goal of Davis and Kahan Q is however different — analyzing the perturbations 
of eigenvalues and eigenspaces, while in the present paper we are concerned with 
sensitivity of angles and Ritz values with respect to changes in subspaces. 

Our second main result, see Section bounds the change in the Ritz values with 
the change of the trial subspace. We attack the problem by discovering a simple, but 
deep, connection between the principal angles and the Raylcigh-Ritz method. 

We first give a brief definition of Ritz values. Let A : H — > H be a Hermitian 
operator and let X be a (so-called "trial") subspace of H. We define an operator 
PxA\x on X, where P x is the orthogonal projector onto X and PxA\x denotes the 
restriction of operator PxA to its invariant subspace X, as discussed, e.g., in Parlett 
Sect. 11.4, pp. 234-239]. The eigenvalues A{P X A\ X ) are called Ritz values of the 
operator A with respect to the trial subspace X. 

We observe that the cosines squared cos 2 @(X,Z) of principal angles between 
subspaces X and Z can be interpreted as a vector of Ritz values, where the Rayleigh- 
Ritz method is applied to the orthogonal projector Pz onto Z using A" as a trial 
subspace. Let us illustrate this connection for one-dimensional X — spanja;} and 
Z = span{z}, where it becomes trivial: 

2/nt \\ \ x -,Pz x ) 

cos'(6(x,z)) = — — . 

(x,x) 

The ratio on the right is the Rayleigh quotient for Pz — the one dimensional analog 
of a Ritz value. In this notation, estimate (|1.4I) turns into 



(x, P z x) (y, P z y) 



(x,x) (y,y) 



<sm(0(x,y)), (1.5) 



which clearly now is a particular case of a general estimate for the Rayleigh quotient, 
cf. Knyazev and Argentati 



(x,Ax) (y,Ay) 



< (Amax - A min ) sin(6(x, y)), (1.6) 



(x,x) (y,y) 

where A is a Hermitian operator and Amax — A mm is the spread of its spectrum. 
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We show that the multi-dimensional analog of i|1.5|) can be interpreted as a 
bound on the change in the Ritz values with the change of the trial subspace, in 
the particular case where the Rayleigh-Ritz method is applied to an orthogonal pro- 
jector. We then extend it to prove a general result for Ritz values for an arbitrary 
Hermitian operator A, not necessarily a projector: let K{PxA\x) be the vector of 
Ritz values in nonincreasing order for the operator A on a trial subspace X , which 
is perturbed to give another trial subspace y, then |A (PxA\x) — A (Pyv4|y)| ~< w 
(Amax — A mm ) sinO(A', y), which is a multi-dimensional analog of Our present 

proof is based on a classical but rarely used idea of extending a Hermitian operator 
in Ti. to an orthogonal projector in the "double" space Ti. 2 , preserving its Ritz values. 

An application of our Ritz values weak majorization result for Laplacian graph 
spectra comparison is suggested in Section \5\ based on the possibility to interpret 
eigenvalues of the edge Laplacian of a given graph as Ritz values of the edge Laplacian 
of the complete graph. We prove that J2k l^fe ~ — n ^ where \\ and X 2 are all 
ordered elements of the Laplacian spectra of two graphs with the same n vertices and 
with I equal to the number of differing edges. 

The rest of the paper is organized as follows. In Section|21 we provide some back- 
ground, definitions and several statements concerning weak majorization, principal 
angles between subspaces, and extensions of Hermitian operators to projectors. In 
Section we prove in Theorems 13.21 and 13.31 that the absolute value of the change in 
(the squares of) the sines and cosines is weakly majorized by the sines of the angles 
between the original and perturbed subspaces. In Section we prove in Theorem 
14.21 that a change in the Ritz values in the Rayleigh-Ritz method with respect to 
the change in the trial subspaces is weakly majorized by the sines of the principal 
angles between the original and perturbed trial subspaces times a constant. In Sec- 
tion[S]we apply our Ritz values weak majorization result to Laplacian graphs spectra 
comparison. 

This paper is related to several different subjects: majorization, principal angles, 
Rayleigh-Ritz method, and Laplacian graph spectra. In most cases, whenever possible, 
we cite books rather than the original works in order to keep our already quite long 
list of references within a reasonable size. 

2. Definitions and Preliminaries. In this section we introduce some defini- 
tions, basic concepts and mostly known results for later use. 

2.1. Weak Majorization. Majorization is a well known, e.g., in Hardy et al. 
[Til pp. 45-49] or Marshall and Olkin [24I pp. 9-14], important mathematical concept 
with numerous applications. 

For a real vector x = [x%, ■ ■ ■ ,x n ] let x^ be the vector obtained by rearranging 
the entries of x in an algebraically non- increasing order, x{ > • ■ • > x^. We denote 
[\xi\, • • • , \x n \] by \x\. We say that vector y weakly majorizes vector x and we use the 
notation [xi, ■ ■ ■ ,x n ] -< w [j/i, • • • ,y n ] or x < w y if J2i=i x\ < £\ =1 yj, k = 
1, . . . , n. If in addition the sums above for k — n are equal, y (strongly) majorizes 
vector x, but we do not use this type of majorization in the present paper. Two 
vectors of different lengths may be compared by simply appending zeroes to increase 
the size of the smaller vector to make the vectors the same length. 

Weak majorization is a powerful tool for estimates involving eigenvalues and sin- 
gular values and is covered, e.g., in Gohberg and Krem Marshall and Olkin j24|, 
Bhatia and Horn and Johnson [l^, which we follow here and refer the reader to 
for references to the original works and all necessary proofs. In the present paper, 
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we use several well known statements that we formulate for operators TL — > TL and 
overview briefly below. 

Let S(A) denote the vector of all singular values of A : TL — > TL in nonincreasing 
order, i.e. S(A) ~ S^A), while individual singular values of A enumerated in non- 
increasing order are denoted by Si(A). For Hermitian A let A(A) denote the vector 
of all eigenvalues of A in nonincreasing order, i.e. A(A) = A^-(A), while individual 
eigenvalues of A enumerated in nonincreasing order are denoted by Xi(A). 

The starting point for weak majorization results we use in this paper is, e.g., 
Marshall and Olkin 0, Th. 9.G.1, p. 241], 

Theorem 2.1. A(A + B) -< w A(A) + A(B) for Hermitian A and B. 
This follows easily from Ky Fan's trace maximum principle Marshall and Olkin 0, 
Th. 20. A. 2, p. 511] and the fact that the maximum of a sum is bounded from above 
by the sum of the maxima. For general A : TL — * TL and B : TL — > TL, it follows 
from Theorem l2.ll since the top half of the spectrum of the Hermitian 2-by-2 block 

operator ® ^ is nothing but S(A), that, e.g., Horn and Johnson 15, Cor. 

3.4.3, p. 196], 

Corollary 2.2. S(A ± B) -< w S(A) + S(B). 
A more delicate and stronger result is the following Lidskii theorem e.g., Bhatia Q, 
Th. III.4.1, p. 69 ], which can be proved using the Wielandt maximum principle, e.g., 
Bhatia 0, Th. III.3.5, p. 67], 

Theorem 2.3. For Hermitian A and B and any set of indices 1 < i\ < ... < 
ik <n = dimTL, we have Y,j=i (A+B) < J2j=i (^)+X^-=i k = l,...,n. 

By choosing an appropriate set of indices, Theorem 12.31 for Hermitian A and B im- 
mediately gives A(A) — A(B) A(A — B), which for singular values of arbitrary 
A :H ->• H and B :H^H is equivalent (see Bhatia Q, Sect. IV.3, pp. 98-101]), to, 
e.g. Horn and Johnson 15, Th. 3.4.5, p. 198] or Bhatia [J Th. IV.3.4, p. 100], 

Corollary 2.4. \S(A) - 5(B) | -< w S(A-B). 
Applying Corollary 12. 41 to properly shifted Hermitian operators, we get 

Corollary 2.5. \A(A) - A(B)\ -< w S(A - B) for Hermitian A and B. 
We finally need the so called "pinching" inequality, e.g. Gohberg and Krem 0, Th. 
II.5.1, p. 52] or Bhatia [1 Prob. II.5.5, p. 50], 

Theorem 2.6. IfP is an orthogonal projector then S(PAP±(I-P)A(I-P)) < w 
S{A). 

Proof. Indeed, A = PAP + (I - P)A(I - P) + PA(I - P) + (I - P)AP so let 
B = PAP + (7 - P)A{I - P) - PA(I - P) - (I- P)AP then (2P - I)A(2P -I) = B, 
where 2P — I is unitary Hermitian, so A* A and B*B are similar and S(A) = S(B). 
Evidently, PAP + (I - P)A(I - P) = (A + B) /2 so the pinching result with the plus 
follows from Corollary 12.21 The pinching result with the minus is equivalent to the 
pinching result with the plus since the sign does not change the singular values on the 
left-hand side: S(PAP ± (I - P)A(I - P)) = S(PAP) U 5((J - P)A{I - P)), since 
the ranges of PAP and (I - P)A(I - P) are disjoint. □ 

2.2. Principal Angles Between Subspaces. Let Px and Py be orthogonal 
projectors onto the subspaces X and y, respectively, of the space TL. We define the 
set of cosines of principal angles between subspaces X and y by 

cosQ{X,y) - [ Sl (P x P y ), ... ,s m (P x P y )], m = min{dimX-,dwiy} . (2.1) 

Our definition (|2.1|) is evidently symmetric: 0(^,3^) = <d(y,X). By definition, the 
cosines are arranged in nonincreasing order, i.e. cos(Q(X ,y)) = (cos(0(A',3^)))''', 
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while the angles 8i(X,y) s [0, 7r/2],i = 1, .. . , m and their sines are in nondecreasing 
order. 

The concept of principal angles is closely connected to cosine-sine (CS) decompo- 
sitions of unitary operators; and we refer the reader to the books Bhatia Stewart 
and Sun [3t| , Stewart [3^ f° r the history and references to the original publications on 
the principal angles and the CS decomposition. We need several simple but important 
statements about the angles provided below. In the particular case dimA" = dim} 7 , 
the standard CS decomposition can be used and the statements are easy to derive. 
For the general case dimA ^ dim}- 7 that is necessary for us here, they can be obtained 
using the general (rectangular) form of the CS decomposition described, e.g., in Paige 
and Saunders [3(|, Paige and Wei For completeness we provide the proofs here 
using ideas from Davis and Kahan |7j, Halmos [13(], preparing our work to be more 
easily extended to infinite dimensional Hilbert spaces. 

We formulate without proofs in Knyazev and Argentati [l^, Th. 3.4, p. 2017 and 
Th. 3.5, p. 2018] statements, equivalent to the theorem below, see also Chatelin 0, 
Ex. 1.2.6]. 

Theorem 2.7. When one of the two subspaces is replaced with its orthogonal 
complement, the corresponding pairs of angles sum up to n/2, specifically: 



.,0 



(2.2) 



where there are maxjdinvt' — dim^; 0} values -k/2 on the left, and possibly extra zeros 
on the right to match the sizes. 

The angles between subspaces and between their orthogonal complements are es- 
sentially the same, 



[(O(x,y))±,o,---,o] = [(®(x ± ,y ± )) i ,o,---,o], 



(2.3) 



where extra 0s at the end may need to be added on either side to match the sizes. 

Proof. Let m 00 = x n y, SDtox = x n y^, rni 1Q = x ± ny, wt n = x- L ny ± ,as 

suggested in Halmos p. 381]. 

Each of the subspaces is invariant with respect to orthoprojectors Px and Py and 
their products, and so each of the subspaces contributes independently to the set of 
singular values oiPxPy in (HI) - Specifically, there are dim9Jtoo ones, dimSJto singular 
values in the interval (0, 1) equal to cos 0(97t o , y), where 9Jl = X n (M 00 © ffloi)- 1 , 
and all other singular values are zeros; thus, 



Q{x,yy = 



-,(e(m ,y))Ko,...,o 



(2.4) 



where there are min {dim(OJloi); dim(9Jlio)} values n/2 and dim(9Jtoo) zeros. 

The subspace 9Jlo does not change if we substitute y^ for y in l|2.4|l , so we have 



,(e(OT ,y i )) i ,o,...,o 



where there are min {dim(SDtoo); dim(37tn)} values n/2 and dim(9Jt i) zeros. Since A 
is an eigenvalue of (PxPy)\m if ancl on ly if 1 — A is an eigenvalue of (PxPy±)\wi , 
we have ^ — 0(9710,3^^) = (©(9Ko,3 ; ))''', and the latter equality turns into 



Q{x,y A 



-,(Q(im ,y))t,o,...,Q 



(2.5) 
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where there are dim(9Jtoi) values n/2, and min {dim(9Jt o); dim(3Jln)} zeros. To 
obtain 1)2.2)) . we make (12. 4|) and l|2.5|) equal by adding max{dim97T i — dim9Jli ; 0} 
values 7r/2 to (I2.4|l and max{dim97too — dim97tii;0} zeros to 1)2.5)) . and noting that, 
since dim(A n y x ) = dimX + dhay X - dim(A + y^), A x n y = (A + and 
dim^- 1 - dim((A + J 7 ^) = dim(A + y^) - dim^, we have dim0Jl O i - dimOJtio = 
dim(A n y x ) - dim(A' ± n y) = dimX + dim^ x - dim(A + y - 1 ) - dim( ( X + y x ) x ) = 
dim A" — dim J 7 . 

The proof above shows that there are dimSJtoo = dim(A ny) zeros on the right in 
(E2J). To prove E2J>, we substitute in (£3 X 1 - for X to get [§, . . . , f , (O^- 1 , 3 7 )) 1 ] = 
[| - 9(A- L ,;y- L ),0, ... ,0] with dim(Af ± n J 7 ) zeros on the right on the one hand 
and exchange <d(X,y) = Q(y,X) in 12. 2|) and then substitute A x for A" to obtain 
[§,...,§, (9(y,A' ± ))^ = [§-e(y,Ar),'0,...,0] with dim(^ n A- 1 ) zeros on the 
right on the other hand. We have equal numbers of zeros on the right in both equalities 
and Q(X^,y) = Q(y, X- 1 ) by the symmetry of our definition (|2.1|) . so subtracting 
both equalities from ir/2 leads to 1)2.3(1 . □ 

We also use the following trivial, but crucial, statement. 

LEMMA 2.8. A((P x Py)\x) = [cos 2 Q(X,y),0, ... ,0], with max{dimA , -dimJ; 0} 
extra 0s. 

Proof. The operator (P x Py)\ x — ({Px Py)(Px Py)*) \x is Hermitian nonnegative 
definite, and its spectrum can be represented using the definition of angles 1)2.1)) . The 
number of extra 0s is exactly the difference between the number dimA of Ritz values 
and the number minjdimA; dimj^} of principal angles . □ 

Finally, we need the following characterization of singular values of the difference 
of projectors, which for dimA = dimj 7 appears, e.g., in Stewart [sH. Th. 5.5.5, p. 43]. 

Theorem 2.9. 

[S(Px - Py), 0, . . . , 0] = [1, . . . , 1, (sin6(A, y), sin 6(A, y)) 1 , 0, . . . , 0], 

where there are |dimA — dinLJ 7 ! extra Is upfront, the set sin 0(A, y) is repeated twice 
and ordered, and extra 0s at the end may need to be added on either side to match the 
sizes. 

Proof. The projectors Px and Py are idempotent, which implies on the one hand 
(Px - Pyf =Px(I- Py) +Py(I~ Px) = P X Py± + PyPx^ , 
so the subspace A is invariant under (P x — Py) 2 - On the other hand, 

(P X - Py) 2 =(I- P X )Py + (I - Py)P X = P x ^Py + Py^P X , 

so the subspace X ± is also invariant under (P x — Py) 2 ■ The projectors P x and Py 
are orthogonal, thus the operator (P x — Py) 2 is Hermitian, and its spectrum can be 
represented as a union (counting the multiplicities) of the spectra of its restrictions 
to the complementary invariant subspaces A and X : 

A((Px-Py) 2 ) = [K{{P x P y ,)\ x ),A{{P x ,Py))\ x A l ■ 
Using Lemma \2. 81 and statement 1)2.2)) of Theorem 12. 71 

[A((p x p y ±)\x),o,'-',o] = [cos a e(*,y- L ),o,...,o] 

= [!,...,!, (sin 2 6(A,y)) i ,0,...,0], 
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where there are max{dimA' — dim^; 0} leading Is and possibly extra zeros to match 
the sizes, and 

[A((iV-Py))U-0,0, . . -,0] = [cos 2 9(A- X ,y), 0, ... ,0] 

= [l,... ) l,(sin 2 e(^,^))^0 ) ...,0], 

where there are max{dim3^ — dimA"; 0} leading Is and possibly extra zeros to match 
the sizes. Combining these two relations and taking the square root completes the 
proof. □ 

2.3. Extending Operators to Isometries and Projectors. In this subsec- 
tion we present a simple and known technique, e.g., Halmos [12l ] and Riesz and Sz.- 
Nagy 35, p. 461] for extending a Hermitian operator to a projector. We give an 
alternative proof based on extending an arbitrary normalized operator B to an isom- 
etry B (in matrix terms, a matrix with orthonormal columns). Glazman and Ljubic 
d Prob. X.1.26, p. 455] and Bhatia 1, Prob. 1.3.6, p. 11] extend B to a block 2-by-2 
unitary operator. Our technique is similar and results in a 2-by-l isomctry operator 
B that coincides with the first column of the 2-by-2 unitary extension. 

Lemma 2.10. Given an operator B : Ti —* Ti with singular values less than or 
equal to one, there exists a block 2-by-l isometry operator B : Ti — > Ti 2 , such that the 
upper block of B coincides with B. 

Proof. B*B is Hermitian nonncgative definite, and all its eigenvalues are bounded 
by one, since all singular values of B are bounded by one. Therefore, I — B*B 
is Hermitian and nonnegative definite, and thus possesses a Hermitian nonncgative 
square root. Let 



B = 



B 

V/ - B*B 



By direct calculation, B*B = B*B + s/I - B*By/I - B*B = I, i.e. B is an isometry. 
□ 

Now we use Lemma |2.10l to extend, in a similar sense, a shifted and normalized 
Hermitian operator to an ortho gona l projector. 

Theorem 2.11. (Halmos yUhl and Riesz and Sz.-Nagy [3b, p. 461]) Given a 
Hermitian operator A : Ti — > Ti with eigenvalues enclosed in the segment [0, 1], there 
exists a block 2-by-2 orthogonal projector A : Ti 2 — ► Ti. 2 , such that its upper left block 
is equal to A. 

Proof. There exists V^4, which is also Hermitian and has its eigenvalues enclosed 
in [0, 1]. Applying Lemma T2.1UI to B — \/~A, we construct the isometry B and set 



A = BB* = 



A 



VT^A 



[ Va ^T~a } = 



A y/A(I-A) 
V A (I - A) I- A 



We see that indeed the upper left block is equal to A. We can use the fact that B is 
an isometry to show that A is an orthogonal projector, or that can be checked directly 
by calculating A 2 = A and noticing that A is Hermitian by construction. □ 
Introducing S = V~A and C = \Jl — A, we obtain 



A 



S 2 SC 
SC C 2 
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which is a well known, e.g., Davis 0, Halmos |l3|. block form of an orthogonal 
projector that can alternatively be derived using the CS decomposition of unitary 
operators, e.g., Bhatia 0, Stewart and Sun [23, Stewart |38j |. 

The importance of Theorem 12 . 1 II can be better seen if we reformulate it as 
Theorem 2.12. (cf. Glazman and Ljubic % Ex. X.27, p. 455]) Given a 
Hermitian operator A : TL — > TL with eigenvalues enclosed in a segment [0,1], there 
exist subspaces X and y in TL 2 such that A is unitarily equivalent to (PxPy)\x > 
where Px and Py are the corresponding orthogonal projectors in TL 2 and \x denotes 
a restriction to the invariant subspace X . 

Proof. We use Theorem 12.111 and take Py — A and Px = g q 

Similar to Theorem l2.12l Lemma 12 . 1 01 implicitly states that an arbitrary normal- 
ized operator B is unitary equivalent to a product of the partial isometry B in TL 2 
and the orthogonal projector in H 2 that selects the upper block in B (called Px in 
the proof of Theorem 12. 12|) . It is instructive to compare this product to the classical 
polar decomposition of B that is a product of a partial isometry and a Hermitian 
nonnegative operator in TL. In Tt 2 , we can choose the second factor to be an orthog- 
onal projector! This statement together with Theorem 12.121 can provide interesting 
canonical decompositions in TL 2 that apparently are not exploited at present, but in 
our opinion deserve attention. 

We take advantage of Theorem l2.12l in the present paper. Using Lemma ETHI with 
<|2.1[1 . Theorem l2 . 1 2l implies that the spectrum of an arbitrary Hermitian operator after 
a proper shift and scaling is nothing but a set of cosines squared of principal angles 
between some pair of subspaces. This surprising idea appears to be very powerful. 
It allows us, in Section to obtain a novel result on sensitivity of Ritz values with 
respect to the trial subspace by reducing the investigation of the Rayleigh-Ritz method 
to the analysis of the principal angles between subspaces that we provide in the next 
section. 

3. Majorization for Angles. In this section we prove the main results of the 
present paper involving sines and cosines and their squares of principal angles, but 
we start with a known statement that involves the principal angles themselves: 

Theorem 3.1. (Qiu et al. 0, Th. 3.2, p. 514]) Let X, y and Z be subspaces 
of the same dimension. Then 

\®(x,z)-®{y,z)\< w Q{x,y). (3.1) 



Theorem 13 . II deals with the principal angles themselves, and the obvious question 
is: are there similar results for a function of these angles, in particular for sines and 
cosines and their squares? For one dimensional subspaces, estimate <|3.1ll turns into 
<|l.l[l that, as discussed in the Introduction, implies the estimate (|1.2ll for the sine. 
According to an anonymous referee, it appears to be known to some specialists that 
the same inference can be made for tuples of angles, but there is no good reference 
for this at present. Below we give easy direct proofs in a unified way for the sines and 
cosines and their squares. 

We first prove the estimates for sine and cosine, which are straightforward gen- 
eralizations of the ID sine (|1.2fl and cosine l|1.3|l inequalities from the Introduction. 

Theorem 3.2. Let dimX = dim^ then 

\sm®(X,Z)-sin®{y,Z)\ < w wxQ{X,y), (3.2) 
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\cosQ(X,Z) - cos 9(^,2)1 -< w sinQ(X,y). (3.3) 

Proof. Let P x , Py and Pz be the corresponding orthogonal projectors onto the 
subspaces X, y and Z, respectively. We prove the sine estimate 1)3.20 . using the idea 
of Qiu and Zhang |33|. Starting with (P x - P z ) - {Py - P z ) = P x - Py, as in the 
proof of the ID sine estimate (|1.2f> . we use Corollary 12. 41 to obtain 

\S{P X - Pz) - S{Py - P Z )\ < w S{P X - Py). 

The singular values of the difference of two orthoprojectors are described by Theorem 
12.91 Since dimA" = dim^ we have the same number of extra Is upfront in S(P X — Pz) 
and in S(Py — Pz) so that the extra l's are cancelled and the set of nonzero entries of 
\S{P X - P z ) - S(Py - P z )\ consists of nonzero entries of | sin9(;f,.Z) - sin9(y, Z)\ 
repeated twice. The nonzero entries of S(P X — Py) are by Theorem 12. 91 the nonzero 
entries of sin 9(^,3^) also repeated twice, thus we come to 1)3. 2)) . 

The cosine estimate 1)3. 3 jl follows directly from the sine estimate 1)3 .2J1 with Z 1 - 
instead of Z because of 1)2.2)1 utilizing the assumption dimA" = dm\V . □ 

In our earlier paper, Knyazev and Argentati |lg, Lem. 5.1, p. 2023 and Lem. 5.2, 
p. 2025] we obtained a particular case of Theorem 13.21 only for the largest change 
in the sine and the cosine, but with improved constants. We are not presently able, 
however, to modify the proofs of Knyazev and Argentati using weak majorization, 
in order to improve the estimates of Theorem l3.2l bv introducing these same constants. 

9ur last, but not least, result in this series is the weak majorization bound for 
the sines or cosines squared, which provides the foundation for the rest of the paper. 

Theorem 3.3. Let dimX — dimy, then 

\cos 2 &(X,Z) - cos 2 9(y,Z)| = | sin 2 Q(X , Z) - sin 2 Q(y, Z)\ -< w smO{X,y). 

Proof. The equality is evident. To prove the majorization result for the sines 
squared, we start with the useful pinching identity 

(Px - Pz) 2 - (Py - Pz) 2 = Pz^ (Px - Py)P z ± - Pz(Px - Py)Pz- 
Applying Corollarv l2.4l we obtain 

|S ((Px - Pz) 2 ) - S ((Py - Pz) 2 ) | -< w S (P z ± (P x - Py)P z ^- - Pz(Px - Py)Pz) ■ 

For the left-hand side we use Theorem l2.9l as in the proof of Theorem 13 .21 except that 
we are now working with the squares. For the right-hand side, the pinching Theorem 
12.61 gives 

S (P Z ±(P X - Py)P z x - Pz(Px - Py)Pz) < w S (P x - Py) 

and we use Theorem 12.91 again to characterize S (P x — Py) the same way as in the 
proof of Theorem 13. 21 □ 
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4. Changes in the Trial Subspace in the Rayleigh Ritz Method. In this 
section, we explore a simple, but deep, connection between the principal angles and 
the Rayleigh-Ritz method that we discuss in the Introduction. We demonstrate that 
the analysis of the influence of changes in a trial subspace in the Rayleigh-Ritz method 
is a natural extension of the theory concerning principal angles and the proximity of 
two subspaces developed in the previous section. 

For the reader's convenience, let us repeat here the definition of Ritz values from 
the Introduction: Let A : TL — * TL be a Hermitian operator and Px be an orthogonal 
projector to a subspace X of TL. The eigenvalues A(PxA\x) are the Ritz values of 
operator A with respect to X, which is called the trial subspace. 

Let X and y both be subspaces of TL and dirrvf = dinO- 7 - The goal of this 
section is to analyze the sensitivity of Ritz values with respect to the trial subspaces, 
specifically, to bound the change |A(P#^4|#) — A(FyA|^)| in terms of sinQ(X,y) 
using weak majorization. Such an estimate is already obtained in Theorem 10 of our 
earlier paper Knyazev and Argentati [l9j by applying Corollary 12.51 to the matrices 
of PxA\x and PyA\y. This approach, however, leads to an extra factor y/2 on the 
right-hand side, which is conjectured in Knyazev and Argentati to be artificial. 

We remove this y/2 factor in our new Theorem O by using the entirely different 
and novel approach: we connect the Ritz values with extension Theorems 12.111 and 
12.121 on the one hand, and with the cosine squared of principal angles in Lemma 12.81 
on the other hand. We have shown in Theorem 12.111 that a Hermitian nonnegative 
definite contraction operator can be extended to an orthogonal projector in a larger 
space. The extension has an extra nice property: it preserves the Ritz values. 

Corollary 4.1. Under the assumptions of Theorem the Ritz values of 

operator A : TL — > TL in the trial subspace X C H are the same as the Ritz values of 
operator A : TL 2 — > TL 2 in the trial subspace 



X = 



X 




CTL = 



TL 




C TL 2 . 



Proof. Let P^ : TL — > TL be an orthogonal projector on the subspace TL and 
P x : TL 2 — > TL 2 be an orthogonal projector on the subspace X . We use the equality 
sign to denote the trivial isomorphism between TL and TL, i.e. we simply write TL = TL 
and X — X . 

In this notation, we first observe that A = P^A\^, i.e. the operator A itself 
can be viewed as a result of the Rayleigh-Ritz method applied to the operator A 
in the trial subspace TL. Second, we use the fact that a recursive application of the 
Rayleigh-Ritz method on a system of enclosed trial subspaces is equivalent to a direct 
single application of the Rayleigh-Ritz method to the smallest trial subspace, indeed, 
in our notation, P-fyP x — Px^n = ^xi smce % GTL, thus 



PxA\ x = (p x P 7l A\ 7i )\_=P x A\ 



x "■ x ' 
□ 

Next we note that Lemma l2~51 states that the Rayleigh-Ritz method applied to an 
orthogonal projector produces Ritz values, which are essentially the cosines squared of 
the principal angles between the range of the projector and the trial subspace. For the 
reader's convenience we reformulate Lemma \'2 . 81 here : Let the Rayleigh-Ritz method 
be applied to A = Pz, where Pz is an orthogonal projector onto a subspace and 
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let X be the trial subspace in the Rayleigh-Ritz method; then the set of the Ritz 
values is A (PxPz\x) — [cos 2 Z), 0, . . . , 0] with max{dinvf — dimZ; 0} extra 0s. 

Now we are ready to direct our attention to the main topic of this section: the 
influence of changes in a trial subspace in the Rayleigh-Ritz method on the Ritz 
values. 

Theorem 4.2. Let A : TL — > TL be Hermitian and let X and y both be subspaces 
of TL and dim X = dirrQ^. Then 



\A(P x A\ x )-A{PyA\ y )\ ^ w (A r 



,) sinG(X,y), 



(4.1) 



where A mm and Amax are the smallest and largest eigenvalues of A, respectively. 

Proof. We prove Theorem 14.21 in two steps. First we show that we can assume 
that A is a nonnegative definite contraction without losing generality. Second, under 
these assumptions, we extend the operator A to an orthogonal projector by Theorem 
12.111 and use the facts that such an extension does not affect the Ritz values by 
Corollary 14. II and that the Ritz values of an orthogonal projector can be interpreted 
as the cosines squared of principal angles between subspaces by Lemma 12.81 thus 
reducing the problem to the already established result on weak majorization of the 
cosine squared Theorem 13. 31 

We observe that the statement of the theorem is invariant with respect to a shift 
and a scaling, indeed, for real a and f3 if the operator A is replaced with (3 (A — a) and 
A mm and Amax are correspondingly updated, both sides of 1)4.1(1 are just multiplied 
by P and 1(4. l|l is thus invariant with respect to a and j3. Choosing a = A mm and 
(3 = l/(A m ax - A mm ), the transformed operator (A - A min )/(A m ax - A mm ) is 
Hermitian with its eigenvalues enclosed in a segment [0,1], thus the statement ((4.1(1 
of the theorem can be equivalcntly rewritten as 



\A{PxA\x)-A(P y A\y)\ < w smQ{X,y), 



(4.2) 



where we from now on assume that A is a nonnegative definite contraction without 
losing generality. 

The second step of the proof is to recast the problem into an equivalent problem for 
an orthogonal projector with the same Ritz values and principal angles. By Theorem 
I2.11l we can extend the nonnegative definite contraction A to an orthogonal projector 
Pg , where Z is a subspace of TL 2 
respect to trial subspaces 



Pg has by Corollary ^4] the same Ritz values with 



X 



X 




C TL 



TL 




C TL and y 



y 
o 



C TL 



H 




C TL 



as A has with respect to the trial subspaces X and y. By Lemma [2.81 these Ritz 
values are equal to the cosines squared of the principal angles between Z and the 
trial subspace X or y possibly with the same number of 0s being added. Moreover, 
the principal angles between X and y in TL 2 are clearly the same as those between 
X and y in TL and dimA" = dimA = dim^ = dirnV. Thus, 1(4.2(1 can be equivalcntly 
reformulated as 

|cos 2 e(A\Z) -cos 2 6(3>,Z))| ^ w sine(X,y). (4.3) 

Finally, we notice that 1(4. 3|) is already proved in Theorem 13.31 □ 

Remark 4.1. As ln -Fnya Z ev and ArgentaU ^, Rema~], th e constant A m ax — 
A mm in Theorem \4-%\ can be replaced with 

max^A'+y, \\x\\=i(x, Ax) - mm xeX +y, \\x\\=i(x, Ax), 
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which for some subspaces X and y can provide a significant improvement. 

Remark 4.2. The implications of the weak majorization inequality in Theorem 
\4-^\ may not be obvious to every reader. To clarify, let m = dimA' = dimj^ and let 
Q!i > • • • > ct m be the Ritz values of A with respect to X and (3\ > • • • > j3 m be the 
Ritz values of A with respect to y. The weak majorization inequality in Theorem \4-.S\ 
directly implies 

k k 
^K-AI 1 < (A m ax- A min )^sin(6 l (A',y)) i , k=l,...,m, 

i=l t=l 

e.g., for k = m we obtain 

m rn 

X>*-A| < (Amax-A min ) £ Bm(Bi(X,y)), (4.4) 

1=1 i=l 

and for k — 1 we have 

Tnaxj=i,...,m\otj - (3j\ < (A max - X m [ n )ga,p(X,y), (4.5) 

where the gap geq)(X,y) between equidimensional subspaces X andy is the sine of the 
larg est angle between X and y. Inequality [4-5\j is proved in Knyazev and Argentati 
0, Th. 5]. 

For real vectors x and y the weak majorization x -< w y is equivalent to the in- 
equality y^.j 4>{xi) < X)"=i fiilJi) f or an V continuous nondecreasing convex real valued 
function cj>, e.g., Marshall and Olkin ]2A. Prop. 4-B.2., p. 109]. Taking, e.g., <f>(t) = t p 
with p > 1, Theorem \4.2\ also implies 

mi m i 

(JT, K - F < (Amax - A min ) ( £ sm(Qi(X, y)f) \ l<p<™. 

i=l i=l 



We finally note that the results of Theorem l4.2l are not intended for the case where 
one of the subspaces X or y is invariant with respect to operator A. In such a case, it 
is natural to expect a much better bound that involves the square of the sin Q(X,y) . 
Majorization results of this kind are not apparently known in the literature. Without 
majorization, estimates just for the largest change in the Ritz values are available, 
e.g., Knyazev and Argentati [19j, Knyazev and Osborn [2l|. 

5. Application of the Majorization Results to Graph Spectra Compari- 
son. In this section, we show that our majorization results can be applied to compare 
graph spectra. The graph spectra comparison can be used for graph matching and 
has applications in data mining, cf. Kosinov and Caelli [23| . 

The section is divided into three subsections. In Subscction l5.ll we give all nec- 
essary definitions and basic facts concerning Laplacian graph spectra. In Subsection 
15.21 we connect the Laplacian graph spectrum and Ritz values, by introducing the 
graph edge Laplacian. Finally, in Subsection 15.31 we prove our main result on the 
Laplacian graph spectra comparison. 

5.1. Incidence Matrices and Graph Laplacians. Here, we give mostly well 
known relevant definitions, e.g., Chung 0], Cvetkovic et al. 0, Merris [26l Wa Eif. 
Mohar [29(, just slightly tailored for our specific needs. 
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Let V be a finite ordered set (of vertices), with an individual element (vertex) 
denoted by Uj £ V. Let E c be the finite ordered set (of all edges), with an individual 
element (edge) denoted by efc S E c such that every efc = [vi,Vj] for all possible 
i > j. E c can be viewed as the set of edges of a complete simple graph with vertices 
V (without self-loops and/or multiple edges). The results of the present paper are 
invariant with respect to specific ordering of vertices and edges. 

Let w c : E c — > R be a function describing edge weights, i.e. w c (ek) € R- If for 
some edge the weight is positive, w c (ek) > 0, we call this edge present, if w c (ek) = 
we say that the edge is absent. In this paper we do not allow negative edge weights. 
For a given weight function w c , we define E C E c such that efc £ E if w c (ek) ^ and 
we define w to be the restriction of w c on all present edges E, i.e. w is made of all 
nonzero values of w c . A pair of sets of vertices V and present edges E with weights 
w is called a graph (V, E) or a weighted graph (V, E, w). 

The vertex-edge incidence matrix Q c of a complete graph (V, E c ) is a matrix 
which has a row for each vertex and a column for each edge, with column-wise entries 
determined as qik = 1, qjk = —1 for every edge efc = i > j in E c and with 

all other entries of Q c equal to zero. The vertex-edge incidence matrix Q of a graph 
(V, E) is determined in the same way, but only for the edges present in E. The vertex- 
edge incidence matrix can be viewed as a matrix representation of a graph analog of 
the divergence operator from partial differential equations (PDE). 

Extending the analogy with PDE, the matrix L = QQ* is called the graph Lapla- 
cian. In the PDE context, this definition corresponds to the negative Laplacian with 
the natural boundary conditions, cf. McDonald and Meyers [25j. Let us note that 
in the graph theory literature such a definitions of the graph Laplacian is usually at- 
tributed to directed graphs, even though changing any edge direction into the opposite 
does not affect the graph Laplacian. 

If we want to take into account the weights, we can work with the matrix 
Q dia,g(w(E))Q* , which is an analog of an isotropic diffusion operator, or we can 
introduce a more general edge matrix W and work with QWQ*, which corresponds 
to a general anisotropic diffusion. It is interesting to notice the equality 

Q c diag( Wc (£ c ))Q c * = Q diag(w(E))Q*, (5.1) 

which shows two alternative equivalent formulas for the graph diffusion operator. 

For simplicity of presentation, we assume in the rest of the paper that the weights 
w c take only the values zero and one. Under this assumption, we introduce matrix 
P = dia,g(w c (E c )) and notice that P is the matrix of an orthogonal projector on a 
subspace spanned by coordinate vectors with indices corresponding to the indices of 
edges present in E and that equality l|5.1|l turns into 

Q C PQ* C - QQ* - L. (5.2) 

Let us note that our results can be easily extended to a more general case of arbitrary 
nonnegative weights, or even to the case of the edge matrix W, assuming that it is 
symmetric nonnegative definite, W — W* > 0. 

Fiedler's pioneering work |8| on using the eigenpairs of the graph Laplacian to 
determine some structural properties of the graph has attracted much attention in 
the past. Recent advances in l arg e-scale eigenvalue computations using multilevel 
preconditioning, e.g., Knyazev [l7| . Knyazev and Neymeyr Koren et al. |2^ . 
suggest novel efficient numerical methods to compute the Fiedler vector and may 
rejuvenate this classical approach, e.g., for graph partitioning. In this paper, we 
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concentrate on the whole set of eigenvalues of L, which is called the Laplacian graph 
spectrum. 

It is known that the Laplacian graph spectrum does not determine the graph 
uniquely, i.e. that there exist isospectral graphs, see, e.g., van Dam and Haemers .'?!) 
and references there. However, intuition suggests that a small change in a large graph 
should not change the Laplacian graph spectrum very much; and attempts have been 
made to use the closeness of Laplacian graph spectra to judge the closeness of the 
graphs in applications; for alternative approaches, see Blondel et al. Q. The goal of 
this section is to backup this intuition with rigorous estimates for proximity of the 
Laplacian graph spectra. 

5.2. Laplacian graph spectrum and Ritz values. In the previous section, we 
obtain in Theorem l4.2l a weak majorization bound for changes in Ritz values depending 
on a change in the trial subspace, which we would like to apply to analyze the graph 
spectrum. In this subsection, we present an approach that allows us to interpret 
the Laplacian graph spectrum as a set of Ritz values obtained by the Rayleigh-Ritz 
method applied to the complete graph. 

A graph (V, E) can evidently be obtained from the complete graph (V, E c ) by 
removing edges, moreover, as we already discussed, we can construct the (V, E) graph 
Laplacian by either of the terms in equality (|5.2|) . The problem is that such a con- 
struction cannot be recast as an application of the Rayleigh-Ritz method, since the 
multiplication by the projector P takes place inside of the product in l|5.2(l . not out- 
side, as required by the Rayleigh-Ritz method. 

To resolve this difficulty, we use the matrix K = Q*Q that is sometimes called 
the matrix of the graph edge Laplacian, instead of the matrix of the graph vertex 
Laplacian L — QQ*, as both matrices K and L share the same nonzero eigenvalues. 
The advantage of the edge Laplacian K is that it can be obtained from the edge 
Laplacian of the complete graph Q*Q C simply by removing the rows and columns 
that correspond to missing edges. Mathematically, this procedure can be viewed as 
an instance of the classical Rayleigh-Ritz method: 

Lemma 5.1. Let us remind the reader that the weights w c take only the values 
zero and one and that P — diag{w c {E c )) is a matrix of an orthogonal projector on 
a subspace spanned by coordinate vectors with indices corresponding to the indices 
of edges present in E. Then Q*Q = (PQtQc)\-R angc (p) , in other words, the matrix 
Q*Q is the result of the Rayleigh-Ritz method applied to the matrix Q*Q C on the trial 
subspace Range(P). The application of the Rayleigh-Ritz method in this case is 
reduced to simply crossing out rows and columns of the matrix Q* C Q C corresponding 
to absent edges, since P projects onto a span of coordinate vectors with the indices 
of the present edges. 

Lemma l5~T1 is a standard tool in the spectral graph theory, e.g., Haemers 0, to 
prove the eigenvalues interlacing; however, the procedure is not apparently recognized 
in the spectral graph community as an instance of the classical Rayleigh-Ritz method. 
Lemma 15.11 provides us with the missing link in order to apply our Theorem El to 
Laplacian graph spectra comparison. 

5.3. Majorization of Ritz Values for Laplacian Graph Spectra Com- 
parison. Using the tools that we have presented in the previous subsections, we now 
can apply the particular case, l|4.4|) . of our weak majorization result of Section 0] to 
analyze the change in the graph spectrum when several edges are added to or removed 



16 



ANDREW V. KNYAZEV AND MERICO E. ARGENTATI 



from the graph. 

Theorem 5.2. Let (V, E 1 ) and (V : E 2 ) be two graphs with the same set of n 
vertices V , with the same number of edges E 1 and E 2 , and with the number of differing 
edges in E 1 and E 2 equal to I. Then 

£|A£-A 2 fe |<nZ, (5.3) 

k 

where X^ and X 2 , are all elements of the Laplacian spectra of the graphs (VjE 1 ) and 
(V, E 2 ) in nonincreasing order. 

Proof. The spectra of the graph vertex and edge Laplacians QQ* and Q*Q are 
the same apart from zero, which does not affect the statement of the theorem, so 
we redefine Ai and X 2 as elements of the spectra, counting the multiplicities, of the 
edge Laplacians of the graphs (V, E 1 ) and (V, E 2 ). Then, by Theorem 15. II and Xj. 
are the Ritz values of the edge Laplacian matrix A = Q*Q C of the complete graph, 
corresponding to the trial subspaces X = Range(Pi) and y — Ranged ) spanned by 
coordinate vectors with indices of the edges present in E 1 and E 2 , respectively. 

Let us apply Theorem 14. 21 taking the sum over all available nonzero values in the 
weak majorization statement as in 1)4. 4[l . This already gives us the left-hand side of 
(|5.3(l . To obtain the right-hand side of 1)5.3(1 from Theorem 14.21 we now show in our 
case that, first, A m ax — ^ m in = n an< ^' secon d, the sum of sines of all angles between 
the trial subspaces X and y is equal to I. 

The first claim follows from the fact, which is easy to check by direct calculation, 
that the spectrum of the vertex (and thus the edge) Laplacian of the complete graph 
with n vertices consists of only two eigenvalues Amax = n and A m j n = 0. Let us 
make a side note that we can interpret the Laplacian of the complete graph as a 
scaled projector, i.e. in this case we could have applied Theorem 13.31 directly, rather 
than Theorem l4.2l which would still result in 1)5.3)1 . 

The second claim, on the sum of sines of all angles, follows from the definition of 
X and y and the assumption that the number of differing edges in E 1 and E 2 is equal 
to I. Indeed, X and y are spanned by coordinate vectors with indices of the edges 
present in E 1 and E 2 . The edges that are present both in E 1 and E 2 contribute zero 
angles into Q(X,y), while the I edges that are different in E 1 and E 2 contribute I 
right angles into <d(X,y), so that the sum of all terms in sinQ(X ,y) is equal to I. □ 

Remark 14.11 is also applicable for Theorem 15.21 — while the min term is always 
zero, since all graph Laplacians are degenerate, the max term can be made smaller 
by replacing n with the largest eigenvalue of the Laplacian of the graph (V, E 1 U E 2 ). 

It is clear from the proof that we do not use the full force of our weak majorization 
results in Theorem l5.2l because it concerns angles which are zero or ir/2. Nevertheless, 
the results of Theorem 15.21 appear to be novel in graph theory. We note that these 
results can be easily extended on fc-partite graphs, and possibly to mixed graphs. 

Let us finally mention an alternative approach to compare Laplacian graph spec- 
tra, which we do not cover in the present paper, by applying Corollarv l2 . 51 directly to 
graph Laplacians and estimating the right-hand side using the fact that the changes 
in I edges represents a low-rank perturbation of the graph Laplacian, cf. So |3j|. 

Conclusions. We use majorization to investigate the sensitivity of angles be- 
tween subspaces and Ritz values with respect to subspaces, and to analyze changes in 
graph Laplacian spectra where edges are added and removed. We discover that these 
seemingly different areas are all surprisingly related. We establish in a unified way 
new results on weak majorization of the changes in the sine/cosine (squared) and in 
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the Ritz values. The main strength of the paper in our opinion is, however, not so 
much in the results themselves but rather in a novel and elegant proof technique that 
is based on a classical but rarely used idea of extending Hermitian operators to or- 
thogonal projectors in a larger space. We believe that this technique is very powerful 
and should be known to a wider audience. 
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